A heuristic for morpheme discovery based on string edit distance

This paper derives from work we have been doing on unsupervised learning of the morphology of languages with rich morphologies, that is, with a high average number of morphemes per word. Our focus in this paper is Swahili, a major Bantu language of East Africa, and our goal is the development of a system that can automatically produce a morphological analyzer of a text on the basis of a large corpus. While a certain amount of work in computational linguistics has already been done on Swahili, our specific goal is a system that can quickly and accurately perform a morphological analysis of any of the approximately 500 Bantu languages when presented with data from it, and little or no computational work currently exists for 99% of them. Our work reported here extends Linguistica, an open source system available at http://linguistica.uchicago.edu.